docs/examples/Make Test Dataset.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "78e3457b",
   "metadata": {},
   "source": [
    "# Make Test Dataset\n",
    "\n",
    "When you encounter unexpected errors in the `power-grid-model`, you would like certainly to report the issue and debug (maybe by another developer) the calculation core with certain dataset. To make this possible, we have implemented a generic mechanism to export/import the dataset to/from JSON files, and to debug the calculation core in both Python and C++ with the test dataset. \n",
    "\n",
    "In this notebook we will learn how test datasets are made in this repository, including:\n",
    "\n",
    "* Structure of validation test datasets in this repository\n",
    "* Format of test datasets (JSON)\n",
    "* Use of helper functions to save and load the datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05a5bd6e",
   "metadata": {},
   "source": [
    "## Structure of Validation Datasets\n",
    "\n",
    "All validation test datasets are located in the [tests/data](https://github.com/PowerGridModel/power-grid-model/tree/main/tests/data) folder. The structure of the folder is as follows:\n",
    "\n",
    "```\n",
    "data\n",
    "   |\n",
    "   |\n",
    "   - power_flow\n",
    "             |\n",
    "             - power_flow_testset_1\n",
    "             - power_flow_testset_2\n",
    "             - ...\n",
    "   - state_estimation\n",
    "             |\n",
    "             - state_estimation_testset_1\n",
    "             - state_estimation_testset_2\n",
    "             - ...\n",
    "```\n",
    "\n",
    "The testsets are separated in two types of calculations: `power_flow` and `state_estimation`. In each folder there are subfolders for individual testset. The test datasets are used in both Python and C++ unit tests. Therefore, once you create extra test datasets in the folder, you can debug the program in both Python and C++."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4315ddc8",
   "metadata": {},
   "source": [
    "### Individual Dataset\n",
    "\n",
    "An individual dataset folder (in either `power_flow` or `state_estimation`) will consists of following files:\n",
    "\n",
    "* `params.json`: calculation parameters, mandatory\n",
    "* `input.json`: input data, mandatory\n",
    "* `sym_output.json`: reference symmetric output\n",
    "* `asym_output.json`: reference asymmetric output\n",
    "* `update_batch.json`: update batch data, mandatory if `sym_output_batch.json` or `asym_output_batch.json` exists.\n",
    "* `sym_output_batch.json`: reference symmetric batch output\n",
    "* `asym_output_batch.json`: reference asymmetric batch output\n",
    "\n",
    "The `params.json` and `input.json` are always needed. The test program (in Python and C++) will detect other files and instantiate relevant test calculations. For example, if `sym_output.json` exists, the test program will run a one-time symmetric calculation and compare the results to the reference results in `sym_output.json`."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "e5460d14",
   "metadata": {},
   "source": [
    "#### Test Parameters\n",
    "\n",
    "The `params.json` looks something like this:\n",
    "\n",
    "```json\n",
    "{\n",
    "  \"calculation_method\": \"iterative_linear\",\n",
    "  \"rtol\": 1e-8,\n",
    "  \"atol\": {\n",
    "    \"default\": 1e-8,\n",
    "    \".+_residual\": 1e-4\n",
    "  }\n",
    "}\n",
    "```\n",
    "\n",
    "You need to specify the method to use for the calculation, the relative and absolute tolerance to compare the calculation results with the reference results. For `rtol` you always give one number. For `atol` you can also give one number, or you can give a dictionary with regular expressions to match the attribute names. In this way you can have fine control of individual tolerance for each attribut (e.g. active/reactive power). In the example it has an absolute tolerance of `1e-4` for attributes which ends with `_residual` and `1e-8` for everything else.\n",
    "\n",
    "The `calculation_method` can be one string or list of strings. In the latter case, the test program will run the validation test mutilple times using all the specified methods."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43d027cd",
   "metadata": {},
   "source": [
    "### JSON Data Format\n",
    "\n",
    "The data format is well explained in these resources\n",
    "[Serialization documentation](../user_manual/serialization.md) and some examples of Serialization are given in [Serialization notebook](./Serialization%20Example.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef0e663c",
   "metadata": {},
   "source": [
    "### Empty Result File\n",
    "\n",
    "If you encounter a crash for a certain dataset. You can also create the input data into JSON files. In this case you might not have any reference result to compare, because you just need to find where the crash happens. You still need an empty (dictionary) result file to trigger the calculation.\n",
    "\n",
    "For `sym_output.json`:\n",
    "\n",
    "```json\n",
    "{\n",
    "  \"attributes\": {},\n",
    "  \"data\": {},\n",
    "  \"is_batch\": false,\n",
    "  \"type\": \"sym_output\",\n",
    "  \"version\": \"1.0\"\n",
    "}\n",
    "```\n",
    "\n",
    "For `sym_output_batch.json`:\n",
    "\n",
    "```json\n",
    "{\n",
    "  \"attributes\": {},\n",
    "  \"data\": [{}, {}, {}],\n",
    "  \"is_batch\": true,\n",
    "  \"type\": \"sym_output\",\n",
    "  \"version\": \"1.0\"\n",
    "}\n",
    "```\n",
    "\n",
    "    \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99220dfe",
   "metadata": {},
   "source": [
    "## Helper Functions to Import and Export\n",
    "\n",
    "In the module `power_grid_model.utils` we have some helper functions to import a json file to a `power-grid-model` compatible dataset, or the other way around. \n",
    "\n",
    "Please refer to the [documentation](../api_reference/python-api-reference.md) for detailed function signature.\n",
    "\n",
    "In this notebook we export the example network from [Power Flow](./Power%20Flow%20Example.ipynb) to json. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b158e92f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# first build the network\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "from power_grid_model import LoadGenType\n",
    "from power_grid_model import PowerGridModel\n",
    "from power_grid_model import initialize_array\n",
    "\n",
    "# network\n",
    "\n",
    "# node\n",
    "node = initialize_array(\"input\", \"node\", 3)\n",
    "node[\"id\"] = [1, 2, 6]\n",
    "node[\"u_rated\"] = [10.5e3, 10.5e3, 10.5e3]\n",
    "\n",
    "# line\n",
    "line = initialize_array(\"input\", \"line\", 3)\n",
    "line[\"id\"] = [3, 5, 8]\n",
    "line[\"from_node\"] = [1, 2, 1]\n",
    "line[\"to_node\"] = [2, 6, 6]\n",
    "line[\"from_status\"] = [1, 1, 1]\n",
    "line[\"to_status\"] = [1, 1, 1]\n",
    "line[\"r1\"] = [0.25, 0.25, 0.25]\n",
    "line[\"x1\"] = [0.2, 0.2, 0.2]\n",
    "line[\"c1\"] = [10e-6, 10e-6, 10e-6]\n",
    "line[\"tan1\"] = [0.0, 0.0, 0.0]\n",
    "line[\"i_n\"] = [1000, 1000, 1000]\n",
    "\n",
    "# load\n",
    "sym_load = initialize_array(\"input\", \"sym_load\", 2)\n",
    "sym_load[\"id\"] = [4, 7]\n",
    "sym_load[\"node\"] = [2, 6]\n",
    "sym_load[\"status\"] = [1, 1]\n",
    "sym_load[\"type\"] = [LoadGenType.const_power, LoadGenType.const_power]\n",
    "sym_load[\"p_specified\"] = [20e6, 10e6]\n",
    "sym_load[\"q_specified\"] = [5e6, 2e6]\n",
    "\n",
    "# source\n",
    "source = initialize_array(\"input\", \"source\", 1)\n",
    "source[\"id\"] = [10]\n",
    "source[\"node\"] = [1]\n",
    "source[\"status\"] = [1]\n",
    "source[\"u_ref\"] = [1.0]\n",
    "\n",
    "# all\n",
    "input_data = {\"node\": node, \"line\": line, \"sym_load\": sym_load, \"source\": source}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73dba42b",
   "metadata": {},
   "source": [
    "### Export to JSON\n",
    "\n",
    "We can use the fuction `json_serialize_to_file` to export the input data to a json file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "724e098a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from power_grid_model.utils import json_serialize_to_file\n",
    "import tempfile\n",
    "from pathlib import Path\n",
    "\n",
    "temp_path = Path(tempfile.gettempdir())\n",
    "json_serialize_to_file(temp_path / \"input.json\", input_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "071c790a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"attributes\": {},\n",
      "  \"data\": {\n",
      "    \"line\": [\n",
      "      {\n",
      "        \"c1\": 1e-05,\n",
      "        \"from_node\": 1,\n",
      "        \"from_status\": 1,\n",
      "        \"i_n\": 1000,\n",
      "        \"id\": 3,\n",
      "        \"r1\": 0.25,\n",
      "        \"tan1\": 0,\n",
      "        \"to_node\": 2,\n",
      "        \"to_status\": 1,\n",
      "        \"x1\": 0.2\n",
      "      },\n",
      "      {\n",
      "        \"c1\": 1e-05,\n",
      "        \"from_node\": 2,\n",
      "        \"from_status\": 1,\n",
      "        \"i_n\": 1000,\n",
      "        \"id\": 5,\n",
      "        \"r1\": 0.25,\n",
      "        \"tan1\": 0,\n",
      "        \"to_node\": 6,\n",
      "        \"to_status\": 1,\n",
      "        \"x1\": 0.2\n",
      "      },\n",
      "      {\n",
      "        \"c1\": 1e-05,\n",
      "        \"from_node\": 1,\n",
      "        \"from_status\": 1,\n",
      "        \"i_n\": 1000,\n",
      "        \"id\": 8,\n",
      "        \"r1\": 0.25,\n",
      "        \"tan1\": 0,\n",
      "        \"to_node\": 6,\n",
      "        \"to_status\": 1,\n",
      "        \"x1\": 0.2\n",
      "      }\n",
      "    ],\n",
      "    \"node\": [\n",
      "      {\n",
      "        \"id\": 1,\n",
      "        \"u_rated\": 10500\n",
      "      },\n",
      "      {\n",
      "        \"id\": 2,\n",
      "        \"u_rated\": 10500\n",
      "      },\n",
      "      {\n",
      "        \"id\": 6,\n",
      "        \"u_rated\": 10500\n",
      "      }\n",
      "    ],\n",
      "    \"source\": [\n",
      "      {\n",
      "        \"id\": 10,\n",
      "        \"node\": 1,\n",
      "        \"status\": 1,\n",
      "        \"u_ref\": 1\n",
      "      }\n",
      "    ],\n",
      "    \"sym_load\": [\n",
      "      {\n",
      "        \"id\": 4,\n",
      "        \"node\": 2,\n",
      "        \"p_specified\": 20000000,\n",
      "        \"q_specified\": 5000000,\n",
      "        \"status\": 1,\n",
      "        \"type\": 0\n",
      "      },\n",
      "      {\n",
      "        \"id\": 7,\n",
      "        \"node\": 6,\n",
      "        \"p_specified\": 10000000,\n",
      "        \"q_specified\": 2000000,\n",
      "        \"status\": 1,\n",
      "        \"type\": 0\n",
      "      }\n",
      "    ]\n",
      "  },\n",
      "  \"is_batch\": false,\n",
      "  \"type\": \"input\",\n",
      "  \"version\": \"1.0\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "# we can display the json file\n",
    "\n",
    "with open(temp_path / \"input.json\", \"r\") as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e85c767",
   "metadata": {},
   "source": [
    "### Import JSON\n",
    "\n",
    "We can use the fuction `json_deserialize_from_file` to import the input data from a json file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "c79d7216",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   id  energized      u_pu             u   u_angle             p             q\n",
      "0   1          1  0.998988  10489.375043 -0.003039  3.121451e+07  6.991358e+06\n",
      "1   2          1  0.952126   9997.325181 -0.026031 -2.000000e+07 -5.000000e+06\n",
      "2   6          1  0.962096  10102.012975 -0.021895 -1.000000e+07 -2.000000e+06\n"
     ]
    }
   ],
   "source": [
    "# round trip and run power flow\n",
    "\n",
    "from power_grid_model.utils import json_deserialize_from_file\n",
    "\n",
    "imported_data = json_deserialize_from_file(temp_path / \"input.json\")\n",
    "\n",
    "pgm = PowerGridModel(imported_data)\n",
    "result = pgm.calculate_power_flow()\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "print(pd.DataFrame(result[\"node\"]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2409710",
   "metadata": {},
   "source": [
    "## Import and Export Batch Update/Result Dataset\n",
    "\n",
    "You can use the same function to import and export batch update/result dataset for batch calculation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "37cd5ade",
   "metadata": {},
   "outputs": [],
   "source": [
    "# create batch set\n",
    "\n",
    "load_profile = initialize_array(\"update\", \"sym_load\", (3, 2))\n",
    "load_profile[\"id\"] = [[4, 7]]\n",
    "# this is a scale of load from 0% to 100%\n",
    "load_profile[\"p_specified\"] = [[30e6, 15e6]] * np.linspace(0, 1, 3).reshape(-1, 1)\n",
    "\n",
    "\n",
    "time_series_mutation = {\"sym_load\": load_profile}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "89011a10",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"attributes\": {},\n",
      "  \"data\": [\n",
      "    {\n",
      "      \"sym_load\": [\n",
      "        {\n",
      "          \"id\": 4,\n",
      "          \"p_specified\": 0\n",
      "        },\n",
      "        {\n",
      "          \"id\": 7,\n",
      "          \"p_specified\": 0\n",
      "        }\n",
      "      ]\n",
      "    },\n",
      "    {\n",
      "      \"sym_load\": [\n",
      "        {\n",
      "          \"id\": 4,\n",
      "          \"p_specified\": 15000000\n",
      "        },\n",
      "        {\n",
      "          \"id\": 7,\n",
      "          \"p_specified\": 7500000\n",
      "        }\n",
      "      ]\n",
      "    },\n",
      "    {\n",
      "      \"sym_load\": [\n",
      "        {\n",
      "          \"id\": 4,\n",
      "          \"p_specified\": 30000000\n",
      "        },\n",
      "        {\n",
      "          \"id\": 7,\n",
      "          \"p_specified\": 15000000\n",
      "        }\n",
      "      ]\n",
      "    }\n",
      "  ],\n",
      "  \"is_batch\": true,\n",
      "  \"type\": \"update\",\n",
      "  \"version\": \"1.0\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "# export and print\n",
    "\n",
    "json_serialize_to_file(temp_path / \"update_batch.json\", time_series_mutation)\n",
    "\n",
    "with open(temp_path / \"update_batch.json\", \"r\") as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "8c4ad876",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[       0.        0.]\n",
      " [15000000.  7500000.]\n",
      " [30000000. 15000000.]]\n"
     ]
    }
   ],
   "source": [
    "# import round trip, calculate\n",
    "\n",
    "imported_batch_update = json_deserialize_from_file(temp_path / \"update_batch.json\")\n",
    "\n",
    "batch_result = pgm.calculate_power_flow(update_data=imported_batch_update)\n",
    "\n",
    "print(batch_result[\"sym_load\"][\"p\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b57f48a0",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.1"
  },
  "vscode": {
   "interpreter": {
    "hash": "39f035c5e58f53e973c1999fdd4393d6db2dd3f04fe05f524988f90f4bbdb576"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}